Hybrid model of Conditional Random Field and Support Vector Machine

Conditional Random Fields (CRFs) [4, 13, 3, 17] are semi-generative (despite often being classified as discriminative models) in the sense that it estimates the conditional probabilityD(y|x) (given any observation x) of any label y, which is generated from D(y|x). Estimating D(y|x) is usually more efficient than estimating D(x|y) when there aren’t sufficient observation x per class or there are too many labels (e.g. there are exponential many y for a chain-like x). To avoid causing terminology confusion, we call the models that estimate underlying distribution ( eitherD(y|x) orD(x|y)) probabilistic models. Unlike CRFs, Support Vector Machine (SVM) is a pure discriminative model in the sense that it seeks for a predicting function regardless of modeling the underlying distribution. We are interested in revealing the nature of probabilistic models and pure discriminative models, in order to obtain a model having the advantages of both.